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Exponential Random Graph Models 

Synonyms 

p* models, p-star models, pi models, exponential family of random graphs, maximum 
entropy random networks, logit models, Markov graphs 



Glossary 

• Graph and network: the terms are used interchangeably in this essay. 

• Real-world network: (real network, observed network) means network data 
the researcher has collected and is interested in modelling. 

• Ensemble of graphs: means the set of all possible graphs (network realiza- 
tions) that the (real-world) network may reasonably be expected to become, 
with the assigned probability distribution, which specifies how likely it is that 
the network will be found in a particular realization. In other words, ensemble 
of graphs is defined by ascribing a statistical weight to every graph in the given 
set. 

• Graph observable: measurable property of a graph. 

• Network Hamiltonian: is a particular type of objective (fitness) function, 
H(G). The exponential random graph model defines a probability distribution 
over a specified set of possible graphs, Q = {G}, such that the probability P(G) 
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of a particular graph G is proportional to e HlyG \ where H(G) = J2i^i^i(G). In 
the Hamiltonian, {xj} is the set of graph observables upon which the relevant 
constraints act, and {6^} is a set of ensemble parameters which we can vary so as 
to match the properties of the model network to the real-world network under 
investigation. 

• Adjacency matrix: is a matrix with rows and columns labelled by graph 
vertices i and j, with elements = 1 or according to whether the vertices, i 
and j, are connected/adjacent or not. In the case of an undirected graph with 
no self-loops or multiple edges (the so-called simple graph), the adjacency 
matrix is symmetric (i.e. Aij = A^) and has Os on the diagonal (i.e. An = 0). 
Accordingly, for a simple directed graph the symmetry condition may not 
be fulfilled, i.e. it can be that A^ ^ Aji. 

• Reciprocity: describes tendency of vertex pairs to form mutual directed con- 
nections between each other. 

• Clustering: describes tendency of nodes to cluster together. Clustering is mea- 
sured by the clustering coefficient which calculates the average probability 
that two neighbors of a vertex are themselves nearest neighbors. 

Definition 

A graph consists of a set of objects or individuals, called nodes (points, vertices), 
connected by links (edges). The idea of a graph is a powerful simplification for different 
phenomena - a way of specifying pairwise relations among a collection of items or 
agents. Graph models are introduced in order to mimic the patterns of connections in 
real networks, in an effort to understand the implications of those patterns, or just to 
describe, how network structures originate, and how they evolve over time. 
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For example, it has been noted that many networks, including social networks, 
have degree distributions that roughly follow a power-law: the so-called scale-free net- 
works. A reasonable question would be to ask how the structure of such networks arises, 
and how the scale-freeness affects their behavior in comparison with the non- scale-free 
counterparts. To address the first question different models of networks' growth and 
evolution have been introduced, such as the famous Barabasi- Albert model of prefer- 
ential attachment. The second question was/is often addressed with the help of ex- 
ponential random graph (ERG) models, which are discussed in this essay. The ERG 
model is particularly useful when one wants to create model networks that match the 
properties of observed networks as closely as possible, but without going into details 
of the specific process underlying network formation. Such model graphs are not only 
interesting in their own right for the light they shed on the structural properties of 
networks. They can be used to study models of processes taking place on networks, 
such as epidemics spreading, diffusion of information, or opinion formation in social 
networks. 

Nowadays, exponential random graphs (ERGs) are among the most widely- 
studied network models. Different analytical and numerical techniques for ERG have 
been developed that resulted in the well-established theory with true predictive power. 
Unfortunately, these advantages come at a price: ERG model is mathematically and 
conceptually rather sophisticated, and its understanding demands some effort of the 
reader. 

Introduction 

Many of the networks we observe in the real world exist in only one realization (instan- 
tiation) that we can study. There is only one Internet, only one World Trade Web, and 
only one network of social ties formed, for example, between filmmakers through their 
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past collaboration in film projects. Only one realization of a given network does not 
mean, however, that this concrete realization is the only possible that the network may 
have. Common sense suggests that in other circumstances different link configurations 
in the considered networks could arise. In particular, the Internet evolves, therefore one 
can see its different structural realizations if one looks at different times. By definition, 
all the snapshots of the Internet are its plausible realizations. It is also reasonable to 
assume that growing in slightly different conditions (we might say, in a parallel world) 
the structure of such an alternative Internet would probably have been similar to the 
real Internet. That is, all plausible realizations of the Internet should have some basic 
features in common, even if they differ in smaller details. Of course, similar considera- 
tions also apply to other types of networks, including economic and financial networks, 
biological networks, and, obviously, social networks. 

Considerations of this kind lead us naturally to the concept of statistical ensem- 
ble of networks, which is the collection of all possible realizations that the considered 
network may reasonably be expected to attain, Q = {G}, plus probability distribution, 
P(G), over Q. In the exponential random graph model each graph, G, appears with 
the probability, P(G) ex e H ^ G \ that is exponential in the so-called graph Hamiltonian, 
H(G), which determines various networks' properties within the ensemble. 

In the following, we show what does it mean "to create" an ensemble of ERGs 
with a given set of properties, such as a given number of edges, or a given value of the 
clustering coefficient \j. We explain the concept of the network Hamiltonian, which is 
the key point of ERG theory. Our theoretical derivations are accompanied by example 
calculations. In particular, the classical random graph model popularized by Erdos and 
Renyi is reformulated in the language of ERGs. Other important examples are also 



1 The clustering coefficient is a measure of the extent to which nodes in a graph tend to cluster 
together [9|. 
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discussed, such as the generalized random graphs, the reciprocity model, the so-called 
two-star model, and Strauss's model of transitive networks. A short training in Monte 
Carlo simulations, to which the ERG model lends itself admirably, is given. Finally, 
we also place an emphasis on deep connections of the model with basic principles of 
equilibrium statistical physics and information theory. In doing so, we argue that these 
models are not merely an ad hoc formulation studied mainly for their mathematical 
convenience, but a true and correct extension of statistical mechanics to the world of 
networks. 

An excellent basic discussion of exponential random graphs addressed to social 



science students and researchers is given in 



26] . This essay is intentionally designed to 



be more theoretical in comparison with the well-known primers just mentioned. Given 
the interdisciplinary character of the new emerging science of complex networks 3], 
the essay aims to give a contribution upon which network scientists and practitioners, 
who represent different research areas, could build a common area of understanding. 



Historical Background 

The first truly general ensemble model of networks was introduced by Solomonoff 
and Rapoport in 1951 [28], who considered the collection of all undirected simple 
graphs with a fixed number of vertices, N, in which every pair of nodes was connected 
with an edge with probability p. In the late 1950s and e arly 1960s, the model was 
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11] . Ever since it is known 



fairly extensively studied by Erdos and Renyi, see e.g. 
as Bernoulli model or Erdos-Renyi model. We mention this because this particular 
ensemble of graphs was indeed the first example of the ERG model, and we will meet 
it again when we will be discussing concrete examples of exponential random graphs. 

The exponential random graph model, as we discuss it in this essay, was first 
proposed in the early 1980s by Holland and Leinhardt [18], who built on statistical 



foundations 
and Strauss 



aid by Bessag 
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5[. Substantial further developments were made by Frank 



29], and continued to be made by other authors throughout 1990s 



!]• 



In recentyears. 



see e.g. 
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a number of physicists have also made theoretical studies in the field, 



24 



15 



20]. 



Nowadays, exponential random grap 
social network analysis (SNA) community 1; 



i models are in common use within the 
. Furthermore, the tool box of standard 



network models/methods is more frequently equipped with the ERG model. Most likely, 
it happens because ERGs are perceived as being a practical tool for modelling any 



complex networks, especially that several standard computer t oo 
simulating and manipulating them, such as the ERGM package 



19] . 



s are available for 



Exponential Random Graphs: Elements of the Theory 

Definition of the Model: Graph Hamiltonian and Ensemble 
Parameters 

Suppose, we have a real- world network and we want to create its ensemble model. For a 
start, we have to define graph observables to be measurable properties of the network, 
that we want to be reflected in the model. Examples of such observables are: the number 
of edges, E, the degree sequence, {ki} = ki, k 2 , . . . k N , the average shortest path length, 
/, and the clustering coefficient, C. In this section, for the purpose of convenience and 
in order to perform general calculations, we assume that the observables, on which 
we focuss, are: {x^ = Xi, x 2 , ■ ■ ■ x r , and their values measured in the considered real 
network are respectively equal to: {x*} 

Now, we have to specify the collection of all possible realizations that our real- 
world network may reasonably be expected to attain. It means, we have to define the 
set of graphs, Q = {G}, that we want to study. In the following, if not stated otherwise, 
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we restrict ourselves to simple graphs with a fixed number of nodes N. A simple graph 
has, at most, one link between any pair of vertices and it does not contain self-loops 
connecting vertices to themselves. There exists a one-to-one correspondence between 
simple graphs and symmetric matrices of size iV with elements Ay equal to either 
or 1 (see Fig. [I]). If we know that the network has N nodes and there is no reason to 
think that iV will change (or change significantly), and if we know that the direction 
of the edges does not matter, then the set of simple graphs is a sensible choice for Q. 




Fig. 1. Statistical ensemble of simple graphs with a fixed number od nodes. 

Thus, we have measurements of one or more graph observables and we have 
also specified plausible realizations of the network. The aim is to choose probability 
distribution P(G) over the set of all possible network realizations, Q = {G}. In the 
exponential random graph model one assumes that the probability distribution, P(G), 
has the forme 

H(G) 

P{G) = (1) 

where Z is called the partition function and can be calculated form the normalization 
condition 

2 The question, of making the best choice of a probability distribution, P(G), given only a relatively 
small number of constraints on that distribution, is addressed in the section Connections with 
Information Theory and Statistical Physics. 



E P(G) = j E e H{G) = 1, (2) 
G&g ^ g&q 

which implies 

Z=Y / e H(G) . (3) 
The network Hamiltonian in Eq. ([1]) is given by 

H(G) = j26MG), (4) 

where {xi(G)} are the values of the observables {x{\ for a graph G, while = 
61,82, ...8 r are ensemble parameters, which are said to be conjugated to observables 
in such a way that the value, x*, as measured in the real network is equal to the 
corresponding average value within the ensemble, i.e. 

(xi) = E Xi(G)P(G) = x*, (5) 
Geg 

for i = 1, 2, . . . r. 

Now, we are in a position to answer the question: What does it mean to define 
an ensemble of exponential random graphs? In short, the ensemble is defined if one 
specifies both: plausible network realizations, Q = {G}, and the observables, {x^, 
which appear in the network Hamiltonian, see Eq. 01]). The ensemble parameters, {8i}, 
used in the Hamiltonian are calculated (analytically or numerically) from the ensemble 
constraints □ given by Eq. (jSJ). 

Properties of the Model: Expectation Values and Their 
Fluctuations 

Once the probability distribution P(G) over Q is given, see Eq. (OQ), we can use it to 
calculate estimates of other quantities of interest within the ensemble. For example, 
the expectation value of a quantity y is given by 



The constraints, Eq. ([S]), consist of r equations in r unknowns. 
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(y) = E y(G)P(G) = \y. y(G)e H{G \ (6) 

Geg " Geg 

This expression provides the best-guess at the value of the quantity y given the only 

general constraints, Eq. In other words, the ERG model enables us to answer 

questions of the form: If we know certain things about a real network, e.g. its measured 

estimates {x*}, what is the best estimate for yl For example, if we know the average 

node degree in a network, what is the best estimate for the clustering coefficient? The 

exponential random graph model gives a rigorous answer to questions of this kind. 

An interesting special case arises when the quantity y is itself one of the pri- 
mary network observables, e.g. y = Xj. One may ask: Why we would want to do this, 
given that, since x* is used as an input to our model, cf. Eq. fl5]), we already know its 
expectation value. The answer is that, to define the ensemble we need to fix the param- 
eters {Oi}. We can do this by calculating expectation values {(xi)} for given ensemble 
parameters, {0i}, and then varying the parameters until {(xi)} take the desired values. 

The value of (xj) within the considered ensemble is given by 

where 

F = \nZ (8) 

is called the free energy, which is a function of r variables {9{\ (alike the partition 
function, Z). 

Similarly to Eq. ([7]), one can show that the second derivative of the free energy, 
F, with respect to 9j gives the mean square fluctuations of Xj, i.e. 

_ (x) , = i*z _ i_ (dz\ 2 d_ n d _z\ = d*F = d(xi 

Vj! \ x i) zd 02 Z^\d9 3 ) de^ZdOj) 86] ' 89 j ' { ) 

In statistical physics, the last expression is known as the fluctuation-response relation. 
The l.h.s. of Eq. ()9]) describes fluctuations in Xj, whereas its r.h.s. characterizes suscep- 
tibility of the observable to its conjugated ensemble parameter 6j. The susceptibility 
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is defined as the derivative of (xj) with respect to 8j, and describes what happens 
with (xj), when one changes its conjugate parameter 9j, which determines/represents 



external conditions related to Xj. 



Connections with Information Theory and Statistical Physics 

Let us define the problem Q Thus, let G be a graph in the set of possible network 
realizations Q and let P(G) be the probability of that graph within the ensemble. We 
would like to choose P(G) so that the expectation value of each of our graph observables 
{xi} = xi,X2, ...x r within the ensemble is equal to its observed value {x*}. Due to 



maximum entropy principle of information theory 



which amounts to the second law 



of thermodynamics in statistical physics 



2], the best choice of probability distribution 



P(G) is the one that maximizes the Shannon/ Gibbs entropy, 

S= - ^2 P(G)lnP(G), (10) 

Geg 

subject to the constraints given by Eqs. and fl5j). 



At this stage, one may as 
In his book on networks (see 



£: What does it mean the best choice in this context? 
20], p. 568) Mark Newman explains that the maximum 
entropy choice is best in the sense that it makes the minimum assumptions about the 
distribution other than those imposed upon us by the constraints. There are choices 
of distribution we could make that would satisfy the constraints but would effectively 
make additional assumptions. For instance, some choices might make a particular graph 
or graphs highly probable while other graphs, only slightly different, are given far lower 
probabilities. These would be considered bad choices in the sense that they assume 
things about the ensemble for which we have no supporting evidence. The entropy as 
given by Eq. ffTO]) is precisely a measure of the amount of assumption that goes into 
a particular choice of distribution P(G), or more precisely it is the amount of antias- 



The discussion here is not used in the sequel and this section can be omitted at the first reading. 
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sumption or ignorance, and by maximizing it we minimize unjustified assumptions as 
much as possible. 

The maximization of the entropy, Eq. (fTOl) . subject to the constraints of Eqs. fl2]) 
and j3J, can be done by the method of Lagrange multipliers. Introducing Lagrange 
multipliers a and {6^} one finds that the maximum value of the entropy, S, is achieved 
for the distribution satisfying the following expression 
d 



dP{G) 



s - a i - £ p(g) - £ *< K - £ ^(gwg) 



Gee 



i=l 



Gee 



0. 



(11) 



for all graphs G E Q. This gives 

r 

- In P(G) - 1 + a + £ 6iXi(G) = 0, 
which implies, cf. Eq. (pQ), 



(12) 



i=l 



P(G) = exp 



a: 



1 + £^(G) 



; H(G) 



(13) 



where i?(G) = Y%=i Qi^iiG) is the graph structural Hamiltonian, Eg. 01]), and Z = e 1 a 
is the partition function, Eq. (j3J). 



Examples 

In the following we discuss five examples illustrating the ERG model. The model of clas- 
sical random graphs, which is also known as the Bernoulli model or Erdos-Renyi model, 
is the first example. It is the simples ensemble of ERGs, and it can be solved exactly, 
meaning that one can calculate its partition function and the corresponding derivatives. 
Next, we discuss generalized random graphs, i.e. maximally random networks with a 
given node degree sequence, which appear to be among the most important network 
models in the contemporary science of complex networks. Over the past decade, they 
have been used extensively as a reliable test bed to study dynamical processes on net- 
works. Then the reciprocity model is worked out, which is an important example of the 
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historical significance in social network analysis and mining. One says that the idea of 
exponential random graphs was born along with the model. It is also worth to note 
that the reciprocity model, like the classical and generalized random graphs, is exactly 
solvable. 

It would be a mistake, however, to think that all or most of exponential random 
graph models can be exactly solved. Most are not, and to make progress one must turn 
to approximate analytic solutions (such as the mean field technique) or numerical cal- 
culations (such as the Monte Carlo methods). The well-known examples of ERGs, for 
which only approximate solutions are known, are the two-star model and the Strauss 's 
model of transitive networks. The two-star model is a toy model. Nevertheless, it is a 
very important example because it illustrates a crucial property of the exponential fam- 
ily. Namely, the two-star model can produce two radically different classes of networks 
for the same values of the ensemble parameters q In physics, the feature is known as 
spontaneous symmetry breaking, and it is understood to accompany phase transitions 
which give rise to interesting phenomena, such as ferromagnetism or superconductivity. 
In the field of social networks, the comprehension of these phenomena is unfortunately 
rather limited. The symmetry breaking is thought to be, at the least, troubling, al- 
though, if presented in the right context, it could be an important /insightful concept 
used, for example, in the theory of social change. 

Classical random graphs 

Suppose, one knows only the expected number of edges, (E), that the undirected 
network has. Thus, the Hamiltonian of the corresponding ERG model is given by 

H(G) = 0E(G), (14) 



5 In the sense of the occurrence probability of a given graph, P(G), it means that the most probable 
network realizations can be completely different. 
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where 9 is the ensemble parameter (so-called external field), whose value determines 
(E). 

In order to proceed, one has to calculate the partition function, Z, of the en- 
semble. When the network Hamiltonian is a simple function of elements Aij(G) of the 
adjacency matrix, as is the case of Eq. (JTlj) above, where 



N N 



N N 



e(g)=j: e 4*(g)=^e e m<*), 
t=i j=i+i z i=i i=i(^i) 



(15) 



then the standard way to perform the sum over Q in Eq. (J2J) is to sum over the elements 
Ay. In the case of simple graphs, the only allowed values for are and 1, with 
An = and A^ = Aji. Hence, the resulting partition function underlying the considered 
ensemble is 



E 



N N 

JE(G) = TJ f[ 

GeG i=l j=i+l 



Gee 

N N 

n n a+ e 

i=l j=i+l 



N N 1 

n n e 

i=l jf=i+l Aij=0 



'l + e^(^ 



(16) 
(17) 



From this expression one can calculate the free energy, cf. Eq. 
F 







> 


1 + e e 





18) 



and the average number of edges is the ERG model, cf. Eq. ([7]), 



(E) 



dF 

~de 



N\ e 6 



(19) 



V 2 j 1 + e 

The last expression can be rearranged to get the value of the only ensemble parameter 
9, behind the given/desired value of (E), i.e. 
(E) 



= ln 



(20) 



At this point, it is insightful to show that the considered ensemble of exponential 
random graphs is equivalent to the so-called binomial model, which is an alternative 
definition of the ensemble of classical random graphs, that was famously studied by 
Erdos and Renyi. In classical random graphs with iV vertices, every pair of nodes is 
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connected with a given probability p. Consequently, the total number of edges is a 
random variable with the expectation value 

(E) = (fjp, (21) 
Note, that the last expression agrees with Eq. ( Fl9i) above, when 

» = TT7- (22 > 

Eq. (I22p shows that there is a direct relationship between the ensemble parameter, 9, in 
the ERG model and the connection probability, p, in the well-known classical random 
graphs. 

The equivalence between the two models becomes even more apparent when 
looking at the probability of obtaining a graph G (with N nodes and E edges) by the 
classical random graph construction procedure, which is 

P(G) =p E (l-p)^h E , (23) 

and the probability of such a graph within the considered ERG model, cf Eq. (JTJ), 

e H(G) BE f e \ E / 1 \("h E 



Z (i + e ^)U) \l + e?J VI + e 
It is easy to see that, given Eq. (|2~2~i) . the two expressions coincide. 

Generalized random graphs 

Despite its general nature and simplicity, the classical random graph model [j turns out 
to have severe shortcomings as a model of real-world networks. Probably the most im- 
portant shortcoming is that the expected node degree distribution in classical random 
graphs, which is the Poisson distribution, significantly differs from the node degree 
distributions in the majority of real networks. To overcome this problem the so-called 



6 which is described in the previous example and shown to be equivalent to the ERG model with an 
expected number of nodes 



15 

generalized random graphs has been proposed, in which degrees of all vertices are drawn 



22] 



from a specified (e.g. power-law/scale- free) degree distribution 

Thus, suppose that, rather than measuring the total number of edges in a net- 
work, we measure degrees of all the nodes. Let us denote by ki the degree of a node i. 
The complete set {ki} = k%, k 2 , . . . kjy is called the node degree sequence of a network^. 
The ERG model appropriate to such a set of observables has the following Hamiltonian 



24] 



H(G)=f2 d MG), (25) 
i=i 

where {9i} is the collection of N ensemble parameters (one parameter 9{ for each node 
i). Noting that ki(G) = J2j=i^ij{G), Eq. (1251) can be rewritten as 

N N N N 

H(G) = EE e iMG) =EEW + 3 )A l3 {G). (26) 

i=l j=l i=l j=i+l 

Accordingly, the partition function for this ensemble is given by, cf. Eqs. ([3]) and ()16p . 

N N N N 1 

z = e n n e^ +e ^^ = n n e e^+°^ ^ 

GeQ i=lj=i+l i=l j=i+l Aij=0 

N N 

= 11 II (l + e^)), (28) 

i=l j=i+l 

and the free energy, Eq. ([8]), becomes 

N N 

F = E E ln [ l + e(9l+ej) ] • ( 29 ) 

i=l j=i+l 

By differentiating the free energy, Eq. (1291) . with respect to #j one gets the 
expression for the average degree of the node j, see Eq. ([7]), 

dF N 1 * 

( ^ ) = ^" = g l + e(^) = g^' (30) 

where 

Pij = ( A ij) = l + e{ e l+ e 3 ) ( 31 ) 



7 Note that, since J^i = we do not need to specify independently the number of edges 
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is the connection probability between the nodes i and j, i.e. the average value of the 
element of the adjacency matrix If. 

In sparse networks, the probability of any individual edge is small, i.e. Pij <C 1. 
In such networks Eq. ( |3T|) factorizes: 



Pij ^ e 



(32) 



Inserting the factorized probability into Eq. (I3"0"j) one gets the following expression for 
the average connectivity of a node j 

N 



(k J )^e- 9 ^^=^ d ^(k)N, (33) 
i=l n 

where the handshaking lemma □ has been used. Then, using Eq. (1331) . the connection 
probability can be written as 

Pv - {k)N ■ W 
The obtained formula for the connection probability between two nodes, % and j, is 
the one which is often encountered in the theoretical description of generalized random 
graphs. 



Reciprocity model 

A large number of real-world directed networks, including social networks, display the 
phenomenon of reciprocity 



30; 



17| . i.e. the tendency of vertex pairs to form mutual 
directed connections between each other ("mutual dyads" in the parlance of social 
network analysis). In this section we discuss the reciprocity model proposed by Holland 



and Leinhardt 



18| 



8 Let us note that Eq. ([51]) can be calculated using Eq. ©. 

2 



9 



N „-6, 
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In the model, the set of all possible network realizations is the set of all simple 
directed graphs o with a fixed number of nodes, N. The network Hamiltonian is given 



by 



H(G) = (3E(G) + 1 R(G), 



(35) 



where E is the number of all directed edges, R stands for the number of vertex pairs 
with edges running between them in both directions, while f3 and 7 are ensemble 
parameters that can be varied to obtain the desired number of all edges, (E), and 
reciprocated edges, (R). Using the elements of the adjacency matrix the Hamiltonian 
can be written as 

N N N N 

H(G) = (3 Y, E MG) + tE E MG)MG) (36) 

i=l j=l i=l j=l 

N N 

= E E \P (MG) + MG)) + M^GH-iCG)], (37) 

i=l j=i+l 

where the product AijAji 7^ only when both directed connections exist, i.e. Aij = 1 
and Aji = 1. 

The partition function of the model is given by |^ 

N N 1 1 N N 

z=n n e ^ j+A]i)+2iAijA]t = u u ( i + 2 ^ + e2W+i) ) (38) 

i=l j=i+l Ay=0 Aji=0 i=lj=i+l 



(l + 2 e^ + e 2 ^)(") ; 



(39) 



and, correspondingly, its free energy is 
_ (N 



In 



1 + 2e ^ + e 2 ^ 



(40) 



Now, using Eq. ([7]), one finds the expected numbers of all edges and reciprocated 
edges, i.e. 

e (S + e 2(/3+ 7 ) 



(F) = ^ = ? N 

K ' dp I 2 ) l + 2e^ + e 2 (^) 



(41) 



and 



10 A simple directed graph is a directed graph having no multiple edges or loops. 

11 cl. detailed calculations shown in Eqs. ([TOf — (JTTJ) 
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r 



1.0 



0.5 



0.0 




-5 



Fig. 2. The fraction of edges that are reciprocated, Eq. in the model of Holland and Leinhardt. 
Note that for varying values of the ensemble parameters, /3 and 7, the reciprocity parameter, r, changes 
from (none of edges belongs to mutual dyad) to 1 (all existing edges are reciprocated). 



One can also calculate the reciprocity parameter, r, which is the fraction of edges that 
are reciprocated: 



From Eqs. (HT1) — (T4"3l . it is evident that it is possible to control both the number of 
edges and the level of reciprocity in the network by suitable choices of the ensemble 
parameters ft and 7 (see Fig. [2]). 

Two-star model 

In the two-star model one specifies the expected number of edges, (E) , and the expected 
numer of the so-called two-stars, (V). Possible network realizations correspond to all 
undirected simple graphs with a given number of nodes, N. A two-star is a vertex 
connected by edges to two other vertices (see Fig. [3]). Varying the number of two-stars 
allows one to control the extent to which edges in the network stick together, meaning 
that the edges share common vertices. If the only number of edges in a network is fixed, 
then sticking together is a random processes, i.e. edges may stick or they may not. On 
the other hand, if we assume a large number of two-stars, then the edges have to stick 




(42) 



(R) _ 1 



(43) 



r = 



(E) 1 + e-(^) ' 
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together to create the required value of (V). In this way, the two-star model allows one 
to control the extent to which the edges gather together in clumps or are distributed 
more randomly. 




Fig. 3. A two-star meaning a vertex connected by edges to two other vertices. 
The number of two-stars in a network is 

N N N 

V(G)=E E E MG)MG) (44) 

-I N N N 

= ~E E MG) E (MG) + A jk (G)) , (45) 
z i=lj=l(/0 k=l&i,j) 

and, therefore, the Hamiltonian of the model is given by 

H(G) = f3E(G) + jV(G) (46) 

i N N / N \ 

= oE E /3 + 7 E (^(G)+A ife (G)) , (47) 

where (3 and 7 are two ensemble parameters, and E(G) was already given by Eq. (11 5p . 

Unfortunately, it is not easy to calculate the partition function, Eq. (j^J), for the 
two-star model. To tell the truth, only approximate solutions of the model are known, 
such as the mean field solution 23|; |2C|. In the following, we present this solution. 
Those, who are not accustomed to analytical calculations may want to skip them, 
but we honestly encourage to be persistent because behavior of the two-star model is 
really crucial. Comprehension of the so-called spontaneous symmetry breaking, that is 
shown to naturally emerge in this model, can be observed in a number of ERG models, 
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including the famous Strauss's model, which is discussed in a more qualitative (i.e. not 
a quantitative) way in the next section. 

Thus, the two-star model can be solved using the so-called mean-field technique, 
which is borrowed from statistical physics. First, let us note that the term Ylk=i&ij) Aik 
in the Hamiltonian, Eq. (I47p . is simply the number of edges attached to vertex i, but 
excluding the connections to i and j. All edges in the model are equivalent so the 
average connection probability, 

P=(Aik), (48) 

is the same for all the pairs of nodes. Therefore, the third sum in Eq. (|47|) can be 
written as 

JV N 

£ (Ai k {G)+A jk {G))~ ((A ik ) + {A jk ))=2(N-2)p N Z 1 2Np, (49) 

fc=i(#,j) k=i(fr,j) 

and, correspondingly, the Hamiltonian of the two-star model becomes equivalent to the 
Hamiltonian underlying classical random graphs, cf. Eq. (I14p . 

i N N 

H{G)~~{(3 + 2 1 N P )Y J E MG) (50) 

i=i j=i&i) 

= ((3 + 2jNp)E(G) (51) 

= 6E(G), (52) 

except that the ensemble parameter in the new Hamiltonian, i.e. 

9 = 13 + 2 7 iV>, (53) 

is a rather complicated function of the original parameters, j3 and 7, and the unknown 
connection probability, p. 

Now, exploiting the observed equivalence, one can use Eq. (fl9|) to calculate the 
average number of edges in the two-star model. Inserting Eq. f[5"B"|) into f fT9l one gets 

W={V) l + e-l>WpV (54) 



i.e. they do not have individual properties to distinguish them 
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Then, equating Eq. ( 154]) to 

(E) = (Tjp, (55) 

where p, is the connection probability, Eq. (|48|) . gives a self-consistent equation for the 

parameter p, which characterizes "density" of connections within the network, i.e. 
1 1 



P 



tanh (-j3 + jNp^j + 1 



1 _|_ e -(/3+2 7 iVp) 2 

In what follows, for convenience in solving for p, we define 



(56) 



B = \^ and C=^N, (57) 
so that Eq. ([55]) becomes 

p = - [tanh(£ + 2Cp) + 1] . (58) 

There is, however, a difficulty with Eq. (158"]) : One does not know its closed- form 
solution. Of course, one can solve this equation numerically, but first it is instructive 
to see the solutions using a graphical method. If we make plots of the lines y = p, l.h.s. 
of Eq. (1381 . and y = |[tanh(5 + 2Cp) + 1], r.h.s. of Eq. (158|) . as functions of p on the 
same axis, they will intersect at the solution (or solutions) of the analyzed equation. 
The method is illustrated in Fig. H] for different choices of the parameters. 

Consider first three plots a), b), and c) which show the r.h.s. of Eq. (158]) (red 
curves) for C = | and three different values of the parameter B. Varying B merely 
shifts the curves horizontally without changing their overall shape. For each curve there 
is a single point of intersection with the line y — p, which is the l.h.s. of Eq. (158]) (blue 
lines), indicated by a small circle. As B is varied this intersection point moves smoothly 
between high and low values of p. It means that for C = | we can tune the density of 
the network to any desired value by the parameter B. 

Now, take a look at three plots g), h), and i) in Fig. HJ which illustrate graphical 
solutions of Eq. (158]) for C = |, and again three different values of B. Like in the 
previous case, varying B shifts the solid curves representing the r.h.s. of Eq. 



22 




Fig. 4. Graphical solutions of p — i[tanh(£? + 2Cp) + 1], i.e. Eq. ((58)) . Depending on the values of the 
parameters B and C the line y = p intersects with the curve y = i[tanh(_B + 2Cp) + 1] either three 
times or only once. Detailed description of the figure is given in the text. 

horizontally, but now there is an important difference. Due to the higher value of C 
the shape of the curves has changed. It is steeper in the middle than it was previously, 
and as a result it is now possible at suitable values of B for the curve to intersect with 
the line, y = p, not just in one place but in three different places. In this regime, there 
are three different possible solutions for p for the same values of the model parameters, 
B and C. The solutions are indicated by small squares. In fact, it turns out that the 
middle solution (open square) is unstable and only the two other are realized in practice. 
These two solutions, however, correspond to very different network realizations. The 
first solution (the smaller value of p) characterizes sparse networks with a few edges, 
whereas the second one (the larger value of p) describes dense networks with many 
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connections. If one were simulate the two-star model on a computer [Jf|, for the same 
parameter values one would in this regime sometimes observe a high-density network 
and sometimes low-density one, depending on initial conditions. In general, one would 
not be able to predict in advance which of these would occur. Worse yet, for C = | 
there are some values of p that are simply impossible to reach. 

This interesting behavior is called spontaneous symmetry breaking. In physics, 
such a behavior is understood to accompany phase transitions. It is known to give rise to 
a number of important phenomena which rely on/emerge from collective behavior of the 
system's constituents. The well-known examples of such phenomena are spontaneous 
ordering or condensation/nucleation effects in the condensed matter physics. More fre- 
quently, however, it happens that researchers from disciplines other than physics strive 
to use these concepts to understand strange/surprising/unexplained observations. It 
seems that the borrowed from physics concepts of symmetry breaking and phase tran- 
sitions may alow understanding properties of systems which are not of interest within 
the traditional domains of physics, but refer to widely circulated research on complex 
systems, which becomes more and more influential. In the SNA community, however, 
the prevailing view is that such a behavior, that a model can produce two radically dif- 
ferent classes of networks for the same values of the model parameters is thought to be 
confusing/false. On the contrary, from the point of view of the physicists, the behavior 
is very intriguing/promising. It suggests, that there exist external conditions at which 
the (social) network becomes very fragile. In the real world, such a fragile/critical net- 



work may abruptly change its properties 



161 ] . One says that the network is susceptible 



to external perturbations. Thus, why not to use the concepts to analyze revolutions, 
political conflicts, etc. 



13 



using, for example, Monte Carlo methods 
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Finally, for the sake of completeness, one should note that the remaining three 
plots d), e) and f) in Fig. H] correspond to the critical value of the parameter C — 1, 
above which the symmetry breaking appears, and below which it is absent. 
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Fig. 5. Phase diagram of the two-star model in (B, C) space. Detailed description of the figure is 
given in the text. 



In Fig. |5l we show the so-called phase-diagram of the two-star model. In the 
diagram, numerical solutions of Eq. 05 8 p are depicted showing different regimes or 
"phases" of the model as a function of its two parameters, B and C. The specific 
solutions of Eq. fl58|) which are indicated in Fig. H]by circles, triangles and squares are 
also adequately plotted in Fig. EJ 



Strauss's model of transitive networks 



The famous Strauss model 29| was originally proposed as a model of a clustered net- 
work. In the model one specifies the expected number of edges, (E) , and the expected 
number of triangles (T) . Consequently, the network Hamiltonian is given by 

H(G) = 6E(G) + aT(G). (59) 

14 i.e. cycles of length 3 
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The Strauss model, like the two-star model, can be solved approximately us- 
ing a mean-field technique 25]. In this case, however, the details of calculation are 
more complicated then for the two-star model. For this reason, we carry out a short 
qualitative discussion of the model, instead of quantitative analysis. 

Thus, there is symmetry breaking in the Strauss model. Phase diagram of the 
model resembles the one, that we have already seen for the two-star model. From the 
diagram, it is evident that there is a structural phase transition in the considered en- 
semble of networks beyond which the system develops a coexistence region where two 
distinct classes of networks can be observed. One class corresponds to high density net- 
works, while the other to low density graphs. In this coexistence region, it also happens 
that no choice of model parameters gives networks of medium density. As a result, there 
are networks that simply cannot be generated by the model. The observation that in 
some circumstances the exponential random graph model does not give desired results 
is a disturbing finding. 

Monte Carlo Simulations of ERGs 

The mathematically tractable models are very rare in the exponential family of ran- 
dom graphs 3 For this reason, in the absence of analytic progress on various mod- 
els (Hamiltonians), researchers have turned to Monte Carlo simulations, a numerical 
method which is ideally suited to exponential random graphs. In what follows we briefly 
describe this method. 

Thus, once the values of ensemble parameters {^} in the Hamiltonian, see 
Eq. (prj), are specified, the form of the probability distribution P(G), Eq. ([I]), makes 
generation of graphs correctly sampled from the ensemble straightforward using a 



15 The solvable examples discussed in the previous section belong to the not very numerous exceptions 
to the rule 
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2]J . In the method, one defines 



Metropolis-Hastings type Markov chain method 
a move-set in the space of graphs and then repeatedly generates moves from this set, 
accepting them with probability 

p=l if P(G')>P(G), (60) 

where G' is the graph after performance of the move, and 

P=T§j if P(G')<P(G), (61) 

while rejecting them with probability 1 — p. Because of the exponential form of P(G), 
the acceptance probability which is given by Eq. (ISTj) is particularly simple to calculate. 
It can be written as 

p = e H(G>)-IHG) =e AH if AH< ^ (62 ) 

where 



AH = Y J i {x l {G , )-x l {G)). (63) 

8=1 

Let us also note, that with the help of AH, the condition for certain acceptance of a 
change, Eq. (160]) . becomes 



p = l if AH > 0. (64) 

The choice of the right move-set first and foremost depends on the set of all 
possible network realizations, Q = {G}, underlying the studied ensemble. For example, 
suitable move-sets are: i) addition and removal of edges between randomly chosen 
vertex pairs for the case of graphs which do not have a fixed number of edges; ii) 
movement of edges randomly from one place to another for the case of fixed edge 
numbers but variable degree sequence; iii) edges swaps of the form {(i>i, Wi), (v%, W2)} — > 
{(i>i,u> 2 ), (^2,^1)} for the case of fixed degree sequence, where (vi,Wi) denote an edge 
from vertex V\ to vertex W\. Monte Carlo numerical simulations of this type are simple 
to implement and appear to converge quickly allowing one to study quite large graphs. 
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To be concrete, let us work out the Metropolis algorithm for classical random 
graphs, i.e. the ERG model with an expected number of edges, cf. Eq. ()14p . The en- 
semble can be obtained in the following way: 

1. At the beginning one creates any simple graph (i.e. its adjacency matrix) with 
a given number of nodes, N. The starting configuration may be, for instance, 
the edgeless graph. 

2. Next, in the following time steps, one randomly chooses a matrix element, 
Aij(G), to be considered for change. For the case, when Aij(G) = 1 (0, re- 
spectively) one considers delation (addition) of the edge, i.e. Aij(G') = (1, 
respectively). This corresponds to the move-set: addition and removal of edges 
between randomly chosen vertex pairs. Whether the change is accepted depends 
on AH. In classical random graphs, since H(G) = 9E(G) one has AH = ±9 
with the upper (lower) sign relating to addition (delation, respectively) of an 
edge. Therefore, the acceptance criteria, Eqs. f )62|) and depend on the sign 
of the ensemble parameter 6, which is given by Eq. ( 120]) . 

3. The updating of elements A^j should be continued until the network observables 
stabilise around their mean values o In the case of classical random graphs, 
the average number of edges, (E), should place itself around the value, which is 
given by Eq. (I15p . Once numerical simulations stabilise, graphs which appear in 
the course of subsequent updates of the adjacency matrix appear to be correctly 
sampled network realizations of the studied ensemble. 



16 



The mean values result from the ensemble parameters. 
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Key Applications and Future Directions 

Bimodality and Symmetry Breaking in Social Networks 

In working with exponential random graph models the aim is to create model networks 
with properties similar to those seen in real-world networks. From the statistical point 
of view, the ensemble approach seems to be a very logical one. The construction of 
the model using maximum entropy principle, which is a driving force behind many 
natural and manmade systems, is natural and ussually gives sensible results. However, 
the overwhelming opinion within the SNA community is that the two features of ERGs, 
i.e. spontaneous symmetry breaking, and the ranges of network properties that simply 
cannot be created using the models, while at the same time real-networks can and do 
display properties in these ranges, indicate that there is a fundamental flaw or gap in 
the reasoning behind ERGs. 

Strauss himself was already aware of these problems when he proposed his model 
of transitive networks in the 1980s, and the fact that the problems with Strauss's model 
are still under active discussion indicates that there is a genuine deadlock here. The is- 
sues must be clarified to make progress in the field of ERG models. To shed light on the 
subject (and, maybe, to start a new debate) let us once again comment on the prob- 
lems with bimodality and symmetry breaking. The SNA community perceives these 
features of ERGs as their serious drawbacks. However, symmetry breaking and phase 
transitions are, in fact, Hamiltonian-dependent inherent features of maximal entropy 
random graphs. If one observes these effects in the network model and at the same 
time one is convinced that the effects are not possible in the real-world network upon 
which the model is originally based, it means that the considered network Hamilto- 
nian is wrong. Imagine also that one can somehow guess the correct Hamiltonian of the 
real-world network. Then the ERG model may allow to make reliable predictions about 
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16|. It is not possible to cheat maximum 



possible/future realizations of this network 
entropy principle and mathematics behind it l£ 

Thus, maybe the problem is not with the ERG model itself, but stems from 
inadequate construction of networks' Hamiltonians? In particular, let us look at the 
Hamiltonian of Strauss's model, Eq. floTJj) . Is it adequate to describe social networks? 
In view of the findings of the science of complex networks, it is not. A major problem 
with this model is that its Hamiltonian is designed for homogeneous systems, in which 
edges and triangles are uniformly distributed over the network. Unfortunately, such a 
homogeneity is not observed in real-world social networks, which are strongly inhomo- 



geneous in terms of the local connectivity and the local clustering 



9[. Therefore, it 



seems that the simples way to overcome the problematic issues with Strauss's model is 



just to refine its Hamiltonian, because mathematics behind the model is correct 



25] 



Predictability and Prediction 

One says that the ultimate proof of our understanding of natural and manmade systems 
is reflected in our ability to predict/controll them. In statistical physics, the problem 
of predictability/controllability is tackled with the help of fluctuation/stability theory 
and response theory - the two pillars on which non-equilibrium statistical mechanics is 
built. 

In physics it is well-understood that interactions between individuals in a sys- 
tem often lead to an emergent global behavior. This behavior is not only imposed 
by external controllers/conditions, but may result from internal interactions/details. 
These issues are closely related to the problem of possible microscopic realizations 



17 In physics, the principle amounts to the second law of thermodynamics. 

18 In particular, one should not forget that exponential random graph models can be only used to 
describe the so-called equilibrium networks. The (real-world) equilibrium network is the one, which 
is not evolving quickly and, therefore, its properties do not change abruptly. 
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of the system o, their macroscopic perception in terms of the system's phases, and 
the corresponding phase transitions (or symmetry breaking phenomena). Furthermore, 
given the ensemble approach to random graphs described in this essay, which directly 
corresponds to ensemble formulation of statistical physics, one can use and modify the 
well-known methods from physics (such as fluctuation-response relations, and Onsager 
reciprocal relations) to apply them to inference on, and analysis of exponential random 
graph models. 

We already know that such generalizations can be done. To be concrete, the 
ideas have been already applied in purely theoretical considerations about ERGs 15]. 



Recently, one has also used them to discuss properties and predictability of real-world 
networks. In particular, having the mathematically tractable yet realistic ERG model of 
the world trade network, one has shown that bilateral trade fulfils a simple fluctuation- 
response theorem describing the susceptibility of trade volume to changes in gross 



domestic products of trade partners 



14 



131 ] . Therefore, one can also believe that the 



analogous quantitative response-like relations may also characterize social networks 
allowing, for example, to predict under which conditions the connected network may 
disintegrate into disconnected clusters. 



Cross- References 

1. Models of Social Networks 

2. Network Models 



3. Scale- free Nature of Social Networks 

19 This is in fact the so-called ensemble formulation of statistical physics 



31 

Acknowledgements 

A.F. acknowledges financial support from the Ministry of Science and Higher Eduction 
in Poland (national three-year scholarship for outstanding young scientists, 2010-2013) 
and from the Foundation for Polish Science (grant no. POMOST/2012-5/5). 

References 

1. Anderson CJ, Wasserman S, Crouch B (1999) A p* primer: logit models for social networks. Social 
Networks 21:37-66 

2. Attard P (2002) Thermodynamics and Statistical Mechanics: Equilibrium by Entropy Maximisa- 
tion. Academic Press, London 

3. Barabasi AL (2002) Linked: The New Science of Networks. Perseus Books Group, Boston 

4. Berg J, Lassig M (2002) Correlated random networks. Phys Rev Lett 89:228,701 

5. Besag JE (1974) Spatial interaction and the statistical analysis of lattice systems. J R Stat Soc 
Ser B Stat Methodol 36:192-225 

6. Burda Z, Krzywicki A (2003) Uncorrelated random networks. Phys Rev E 67:046,118 

7. Burda Z, Correia JD, Krzywicki A (2001) Statistical ensemble of scale-free random graphs. Phys 
Rev E 64:046,118 

8. Cover TM, Thomas JA (1991) Elements of Information Theory. John Wiley, New York 

9. Dorogovtsev SN (2010) Lectures on Complex Networks. Oxford University Press, Oxford 

10. Erdos P, Rcnyi A (1959) On random graphs. Publicationes Mathematicae 6:290-297 

11. Erdos P, Renyi A (1960) On the evolution of random graphs. Publications of the Mathematical 
Institute of the Hungarian Academy of Sciences 5:17-61 

12. Frank O, Strauss D (1986) Markov graphs. J Am Stat Assoc 81:832-842 

13. Fronczak A (2012) Structural hamiltonian of the international trade network. Acta Phys Pol B 
Proc Suppl 5(l):31-46 

14. Fronczak A, Fronczak P (2012) Statistical mechanics of the international trade network. Phys Rev 
E 85:056,113 

15. Fronczak A, Fronczak P, Holyst JA (2006) Fluctuation-dissipation relations in complex networks. 
Phys Rev E 73:016,108 



32 

16. Fronczak P, Fronczak A, Holyst JA (2007) Phase transitions in social networks. Eur Phys J B 
59:133 

17. Garlaschclli D, Loffredo MI (2004) Patterns of link reciprocity in directed networks. Phys Rev 
Lett 93:268,701 

18. Holland PW, Leinhardt S (1981) An exponential family of probability distributions for directed 
graphs. J Amer Stat Assoc 76:33-50 

19. Hunter DR, Handcock MS, Butts CT, Goodreau SM, Morris M (2008) Ergm: A package to fit, 
simulate and diagnose exponential- family models for networks. J Stat Softw 24:1-29 

20. Newman MEJ (2010) Oxford University Press, Oxford, pp 565-588 

21. Newman MEJ, Barkema GT (1999) Monte Carlo Methods in Statistical Physics. Clarendon Press, 
Oxford 

22. Newman MEJ, Strogatz SH, Watts DJ (2001) Random graphs with arbitrary degree distributions 
and their applications. Phys Rev E 64:026,118 

23. Park J, Newman MEJ (2004) Solution of the two-star model of a network. Phys Rev E 70:066,146 

24. Park J, Newman MEJ (2004) Statistical mechanics of networks. Phys Rev E 70:066,117 

25. Park J, Newman MEJ (2005) Solution for the properties of a clustered network. Phys Rev E 
72:026,136 

26. Robins G, Pattison P, Kalish Y, Lusher D (2007) An introduction to exponential random graphs 
(p*) models for social networks. Social Networks 29:173-191 

27. Snijders TAB (2002) Markov chain monte carlo estimation of exponential random graph models. 
Journal of Social Structure 3(2) 

28. Solomonoff R, Rapoport A (1951) Connectivity of random nets. B Math Biophys 13:107-117 

29. Strauss D (1986) On a general class of models for interaction. SIAM Rev 28:513-527 

30. Wasserman S, Faust K (1994) Social Network Analysis. Cambridge University Press, Cambridge 



